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ABSTRACT 


Att  application  of  iemi-Markov  chains  to  Anti  Submarine  Warfare  (ASW)  tactical 
systeiTiS  is  illustrated  with  a  hypothetical  exa-apte.  The  example  involves  estimating  the 
probabilities  cf  prosecuting  false  contacts  for  varying  lengths  of  time.  The  use  of  Markov 
chains  in  the  analysis  of  ASV/  systems,  using  Fleet  ASW  Data  Analysis  Program  (FADAP) 
data,  is  discussed  bnefiy. 


(REVERSE  BLANK) 


INTRODUCTION 


In  measuring  the  '>enormance  of  an  ASW  system,  one  of  the  important  variables  is  the 
degradation  in  performance  caused  by  the  prosecution  of  false  contacts.  Also,  if  the  system 
is  degraded,  is  it  because  the^e  is  a  high  frequency  of  false  contacts,  or  is  it  because  the  false 
contacts  are  prosecuted  ior  excess!  'e  lengths  of  timeTThe  whole  sequence  of  events  from 
detection  to  attack  by  ASW  units  i.^  often  analyzed  by  describing  the  sequence  with  Markov 
cha'us  as  in  reference  (a).  It  is  the  purpose  of  this  paper,  first,  to  point  out  that  the  use  of 
certain  hdarkov  chains  can  be  extended  in  a  natural  way  to  include  waiting  times  in  the 
variou--'  stales.  The  waiting  time  in  a  state  described  as  “prosecution  of  a  false  contact”  is 
interpreted  as  the  time  spent  prosecuting  the  false  contact.  Second,  the  structure  of  the 
FADAP  data  bank  is  such  that  :t  describes  the  ASW  events  sequentially  and,  therefore,  lends 
itself  to  analysis  of  qus.jtions  such  as  those  pos;d  above  by  means  of  Markov  chains. 

DEFINITIONS  AND  ^OTATIONS 

Suppose  M  denotes  a  Markov  chain  with  a  sequence  of  random  variables  Mj.  . 
Associated  with  M  is  a  mUMj',  ?  of  fansition  probabilities.  The  transition  probabilities 
are  d  cnoted  by 


Pir  =  -  r  jM,^  =  i} 


where  Pir  is  the  probability  the  proc.:.ss  wtii  be  in  state  r  on  k+I  step  given  that  the 
process  is  in  state  i  on  step  k  .  Nosv  suppose  that  0  and  the  time  spent  in  state  Mj^  is 
defined  by  tj^+j-  t|j.  A  sequence  {’‘tj  where  .v.  Mj^  provided  that  tjj<t<tjj+j  is  called 
a  semi-Markov  chain.  References  ( h)  and  (c)  define  the  substate 
chain  |(Xf,y{'[  where  yj  =  0  if  t  =  tjjand  y  =  t}c+j  -  t  if  t<tj^+j  and  show  that 
this  substate  chain  is  also  a  Markov  chain.  A  typic.’l  sequence  in  the  substate  chain  might 
appear  as  follows: 

(i,0),  (i,5),  (i,4),  (i,3),  (i,2),  (i,l),{j>0), 

(k.0),(m,0),(m,3),... 

Let  Wjj  =  Pr  {tj^+j  -  tjj  =  j  I  Mj;  •}  •  This  is  the  distribution  o*'  durations  of  system 
stale  i.  Only  integer  durations  are  considered,  a  isf.  ctoi"y  ’ipproximation  if  an 
appropriate  time  scale  is  chosen.  The  following  notation  is  adopted. 


00 

^ij  "  ^  '^in  '  {the  probability  the  duration  in  state  I  is  greater  ina:i  j) 

n=j  +  i 

C9 

“i  =  2  jw:-  ,  (the  average  duration  in  state  i). 

j  =  0  ^ 

The  long  term  expectations  of  being  in  state  i  and  sub-state  (i  j)  are,  respectively, 

nij  =  liin  I’r  =  .and 
k-+oo 

mjj=  lini  Pr  |{x^,yj)  =  (ij)j  . 

1— *co 

References  (bl  and  (c)  show  that 

/n 

mjj  =  bijnij  /  2  e„  , 

/  n=  j 

OB 

provided  bjj>0  ,M  is  apericxlic  and  non-null,  and  j  ^n^’n  con”erges.  N  is  the  total 
number  of  states.  To  say  that  M  is  aperiodic  means  every  state  in  M  can  be  returned  to  in 
any  number  of  steps  greater  than  one.  To  say  that  is  non-null  means  that  each  mj>0  . 

In  order  for  the  steady  state  prooabiiiiics,  »*i  to  be  unique,  M  must  be  an 
irreducible  Markov  chain,  i.e.,  all  pairs  of  states  of  the  chain  must  communicate.  Thus,  it 
must  be  possible  to  arrive  at  any  state  r  from  any  other  state  s  in  a  finite  number  of  steps. 

It  can  be  shown  that  one  characteristic  of  the  steady  state  probabilities  mj  is  that 
(mj  .m2 - -  mjg)  P  =  (mj,m2, . -.mjsj) 

where  P  is  the  matrix  of  transition  probabilities  mentioned  above.  This  fact  is  used  to 
determine  the  m^’s  in  the  application  that  follows. 

The  interpretation  of  mjj  is  dis:ussed  below. 
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APPLICATION 

The  ststes  of  a  Markov  chain  M  -Jescribing  the  events  in  an  ASW  engagement  might  be 
defined  as  follows; 


holding  no  contact. 

holding  a  valid  contact,  but  not  prosecuting  it, 
holding  a  false  contact,  but  not  prosecuting  it, 
prosecuting  a  valid  contact,  and 
prosecuting  a  false  contact. 


It  IS  assumed  that  the  ASW  engagement  goes  on  through  these  five  states  indefinitely.  (This 
is  equivalent  to  assuming  that  the  chain  is  non-null.)  This  situation  may  be  approximated 
when  one  ASW  aircraft  relieves  another  on  a  bajtier  station.  It  is  submitted  that  the 
computation  of  m5i  is  of  interest.  This  number  represents  the  probability  density  of  being 
in  state  (5  j)  in  the  substate  chain.  For  this  application,  it  is  the  long  term  expectation  of 
prosecuting  a  false  contact  for  a  length  of  time  j.  It  is  necessary  that  transition  probabilities 
oi  P  be  estimated  so  that  each  mj  can  be.  computed.  These  are  then  computed  from  the 
following  sets  of  equations; 


inj  -  I  mj  pjj .  and 
i  =  1 


2  mj  =  1. 

i  =  i 


Further  it  is  necessary  to  estimate  the  distributions  of  Wjj  .  in  order  for  the  (constant)  set 
of  transition  probabilities  Pjj  and  the  waiting  t*  ne  distribution  Wjj  to  have  operational 
meaning,  a  constant  submarine  density  or  constant  arrival  rate  must  be  assumed.  It  is  not 
necessary  tc  compute  this  density  or  arrival  rate,  however. 


AN  EXAMPLE 

Suppose,  waiting  times  in  the  five  states  are  retrieved  and  displayed  as  in  table  I,  which 
shows,  for  example,  that  there  were  3,096  occasions  when  no-contact  was  held  for  1  to  10 
minutes.  Further,  suppose  that  the  frequencies  of  transition  from  state  to  state  were  as 
shown  in  figure  1.  For  example,  state  1.  holding  no  contact,  was  followed  9,288  times  by 
state  2,  holding  a  valid  contact,  and  3,096  times  by  state  3,  holding  a  false  contact.  Hence 
the  following  transition  probabilities  are  computed; 
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HYPOTHETICAL  OBSERVED  WAITING  TIMES 


Bggema  Cl 


goes  as  follows.  First,  the  nijj’sare 


The  computation  of  ni5j  =  b^j  mJ  2  e^mj^ 
calculated  from:  /  n=l 

m  1  +  ir2  +  015  +  ni4  +  mg  =  1  , 

11., 

mj  =  0  ■  mj  +  T  •  m2  +  3  •  m3  +  1  •  m4  +  1  *  mj  , 

mo  =  ^•mj+Om',  +  0-  m3  +  0  •  m4  +  0  •  m^  , 

m3  =  ^•mj+0m-)  +  0m3  +  0-  m4  +  0  •  m5  , 

m4  +  0‘m|+i  mo  +  O-  m3  +  0  ■  m4  +  0  •  m^  ,  and 

mj  =  0  •  m  j  +  0  ■  mo  +  j  •  m3  +  0  •  m4  0  •  m^  . 


The  result  is: 


mj  =  =  0.393, 

m2  =  =  0.295, 

^  61 

m3  =  -A  =  0.098, 

6i 

m4  =  ^  =  0.148,  and 

61 


m5  =  -i  =  0.066 
61 


If  the  occurrences  are  assumed  to  be  equally  distributed  among  the  mim.tes  in  each 
ten-minute  interval  of  table  I,  the  e^’s  may  be  computed  as  follows- 

e  j  =  (0)  (99 1  / 1 2384)  -h  ( 1 )  (309.6/ 1 2384)  +  ( 2^  (309.6/ 1 2384) 

-r...  +  (21)  (185.8/12384)+...+  80(6.2/12384). 

The  values  63,  €4,  and  e5  are  computed  in  a  similar  maunur.  The  result  of  these 


computations  is: 


e,  =  20.66 
e2  =  20.66 
e3  =  15.50 
64  =  15.50,  and 
ej  =12.17, 


/5 

so  that  nij  /  -  0.066/18.83  =  0.0035.  Now  it  is  necessary  only  tc 

compute  0.0035 b5j  for  each  waiting  time  j  from  1  to  30  (since  b5j  =  0  for  j  30).  For 
example: 

b5  28  =  (34.4/2064) +  (.34.4/2064).  and 
me  Tg  ~  Q.0035  be  28  ^  O.uOui  /5  . 

The  probability  curve  for  mjj  is  plotted  in  flgure  3. 

Suppose,  that  under  the  conditicris  to  which  the  above  computations  apply,  it  is 
estimated  that  the  ASW  sysiem  is  seriously  degraded  by  the  prosecution  of  a  false  contact 
only  if  the  prosecution  time  exceeds  10  minutes.  The  probability  that  a  false  contact  is 
being  prosecuted  and  will  continue  to  be  prosecuted  for  10  more  minutes  would  be 
computed  by  summing  m5j  over  j  equal  to  or  greater  than  10.  In  this  case 

30 

2  me:  =  0.0174. 
j=  10 

Note  that  this  is  not  to  be  confused  with  the  probability  that  a  false  contact  is  being 
prosecuted  and  that  the  total  prosecution  time  will  be  10  minutes  or  more.  This  latter 
probability  in  general  will  be  larger,  iiid  in  this  case  is  about  0.03. 


3 
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if 


ll 


i 


I  It  should  be  pointed  out  that  in  practice  one  seldom  has  available  as  large  a  set  of  data 

:  in  one  fo.perimeni  as  shown  in  the  hyiX>thetical  example  of  table  I.  Typically  the  waiting 

1  time  vi'stribution  in  one  state  is  estimated  from  one  experiment  and  the  waiting  time 

;  distribution  for  another  state  from  another  experiment.  If  this  were  not  the  case,  the 

I  required  probabilities  could  be  computed  by  taking  the  appropriate  sums  rnd  ratios  directly 

I  from  table  I. 

I 

!  FADAPDATA 

I 

\  The  transition  probabilities  can  be  estimated  directly  from  FADAP  data  by  simply 

1  observing  the  relative  frequencies  of  rne  vsrious  transitions  as  was  done  in  the  example.  One 

is  restricted  only  ia  detining  states  that  correspond  to  events  as  defined  by  reference  (d). 
Examples  of  events  include  initial  detection,  no  detection,  false  contact,  regain  contact, 
additional  target  detection,  confirmation,  localization,  attack,  change  in  contact 
classification,  and  target  exposure  change. 

I 

I  The  data  bank  is  structured  so  that  many  waiting  time  distributions,  Wj|’s,  can  be 

•  estimated.  There  are  provisions  for  obtaining  th".  times  that  many  events  are  initiated  and 

i  terminated. 


LIMITATIONS 


One  limitation  to  the  substate  chain  model  is  the  range  of  choices  for  the  states  of  the 
Markov  chain  M,  The  model  does  not  allow  for  absorbing  states,  which  are  often  very 
natural  for  describing  ASW  attack  sequences.  An  absorbing  state  is  a  state  that  cannot  be 
left  once  it  has  been  entered Pii-=0 when  if  r).  However,  a  way  of  handling  waiting  times 
.vhen  absorbing  states  are  necessary  is  described  in  reference  (e). 

Most  of  the  usual  limitations  of  data  coiiecticn  apply.  One  major  nuisance  is  missing 
data.  A.nother  is  that  the  very  process  of  carefully  defining  states  excludes  so  many  data  that 
sample  sizes  chronically  aie  small.  To  a  degree  this  is  being  overcome  by  the  continual 
addition  of  data  from  fleet  exercises  to  the  data  bank.  However,  since  different  exercises 
test  different  hypotheses,  some  exercises  may  not  include  all  the  states  defined  by  a  chain. 
Also,  equipment  changes  .'ver  the  years  can  affect  both  transition  probabilities  and  waiting 
firaes. 

CONCI.OSIONS 

Despite  the  limitations  cited  above,  it  is  felt  that  the  FADAP  data  bank  provides  data 
structured  in  such  a  way  that  meaningful  analysis  can  be  achieved  by  the  use  of  Markov 
choms.  Further,  in  some  cases  Markov  chains  can  be  extended  in  such  a  way  to  account  for 
waiting  times  in  various  states.  In  particular,  it  is  felt  that  the  probability  that  a  false  contact 
will  be  prosecuted  for  a  given  time  conditional  only  on  the  structure  of  the  whole  ASW 
sequence  of  events  can  be  computed. 
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